Speed up tidy #62037

Mark-Simulacrum · 2019-06-21T21:23:56Z

master:
Time (mean ± σ): 3.478 s ± 0.033 s [User: 3.298 s, System: 0.178 s]
Range (min … max): 3.425 s … 3.525 s 10 runs

This PR:
Time (mean ± σ): 1.098 s ± 0.006 s [User: 783.7 ms, System: 310.2 ms]
Range (min … max): 1.092 s … 1.113 s 10 runs

Alleviates #59884. For the most part each commit stands on its own. Timings are on warm filesystem cache.

r? @eddyb

rust-highfive · 2019-06-21T22:54:35Z

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:00978e6b:start=1561152293273614327,finish=1561152294076285515,duration=802671188
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---
travis_time:start:test_assembly
Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:03:56] 
[01:03:56] running 9 tests
[01:03:56] iiiiiiiii
[01:03:56] 
[01:03:56]  finished in 0.150
[01:03:56] travis_fold:end:test_assembly

---
travis_time:start:test_debuginfo
Check compiletest suite=debuginfo mode=debuginfo-gdb+lldb (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:04:12] 
[01:04:12] running 122 tests
[01:04:36] .iiiii...i.....i..i...i..i.i.i..i.ii..i.i.....i..i....i..........iiii..........i...ii...i.......ii.i 100/122
[01:04:41] .i.i......iii.i.....ii
[01:04:41] 
[01:04:41]  finished in 28.924
[01:04:41] travis_fold:end:test_debuginfo

---
travis_time:start:stage0-unstable-book-gen
Building stage0 tool unstable-book-gen (x86_64-unknown-linux-gnu)
[01:29:22]    Compiling num-traits v0.2.6
[01:29:24]    Compiling unstable-book-gen v0.1.0 (/checkout/src/tools/unstable-book-gen)
[01:29:24] error[E0432]: unresolved import `tidy::features::collect_lib_features`
[01:29:24]  --> src/tools/unstable-book-gen/src/main.rs:8:41
[01:29:24]   |
[01:29:24] 8 | use tidy::features::{Feature, Features, collect_lib_features, collect_lang_features};
[01:29:24]   |                                         |
[01:29:24]   |                                         no `collect_lib_features` in `features`
[01:29:24]   |                                         help: a similar name exists in the module: `collect_lang_features`
[01:29:24] 
---
travis_time:end:042fd443:start=1561157671392699414,finish=1561157671397850919,duration=5151505
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:03ccfe64
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:0784c58c
travis_time:start:0784c58c
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:02da1b9e
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

eddyb · 2019-06-23T01:18:29Z

@bors r+

bors · 2019-06-23T01:18:31Z

📌 Commit 2a0d1cc4182d6cb3433175f3b8ace42e3d99dec3 has been approved by eddyb

bors · 2019-06-23T03:09:12Z

🔒 Merge conflict

This pull request and the master branch diverged in a way that cannot be automatically merged. Please rebase on top of the latest master branch, and let the reviewer approve again.

How do I rebase?

Assuming self is your fork and upstream is this repository, you can resolve the conflict following these steps:

git checkout tidy-fast (switch to your branch)
git fetch upstream master (retrieve the latest master)
git rebase upstream/master -p (rebase on top of it)
Follow the on-screen instruction to resolve conflicts (check git status if you got lost).
git push self tidy-fast --force-with-lease (update this PR)

You may also read Git Rebasing to Resolve Conflicts by Drew Blessing for a short tutorial.

Please avoid the "Resolve conflicts" button on GitHub. It uses git merge instead of git rebase which makes the PR commit history more difficult to read.

Sometimes step 4 will complete without asking for resolution. This is usually due to difference between how Cargo.lock conflict is handled during merge and rebase. This is normal, and you should still perform step 5 to update this PR.

Error message

warning: Cannot merge binary files: Cargo.lock (HEAD vs. heads/homu-tmp)
Auto-merging Cargo.lock
CONFLICT (content): Merge conflict in Cargo.lock
Automatic merge failed; fix conflicts and then commit the result.

It's more efficient than the fs::read_dir API

Mark-Simulacrum · 2019-06-23T13:11:04Z

@bors r=eddyb

bors · 2019-06-23T13:11:05Z

📌 Commit 777951c has been approved by eddyb

@eddyb

Speed up tidy master: Time (mean ± σ): 3.478 s ± 0.033 s [User: 3.298 s, System: 0.178 s] Range (min … max): 3.425 s … 3.525 s 10 runs This PR: Time (mean ± σ): 1.098 s ± 0.006 s [User: 783.7 ms, System: 310.2 ms] Range (min … max): 1.092 s … 1.113 s 10 runs Alleviates rust-lang#59884. For the most part each commit stands on its own. Timings are on warm filesystem cache. r? @eddyb

@ghost

Rollup of 5 pull requests Successful merges: - #61778 (compiletest: Introduce `// {check,build,run}-pass` pass modes) - #62037 (Speed up tidy) - #62052 (submodules: Update clippy from 5a11ed7 to c5d1ecd) - #62070 (Run rustfmt on some libsyntax files) - #62075 (Remove `ast::Guard`) Failed merges: r? @ghost

bors · 2019-06-23T20:05:07Z

⌛ Testing commit 777951c with merge de7c4e4...

@eddyb

Speed up tidy master: Time (mean ± σ): 3.478 s ± 0.033 s [User: 3.298 s, System: 0.178 s] Range (min … max): 3.425 s … 3.525 s 10 runs This PR: Time (mean ± σ): 1.098 s ± 0.006 s [User: 783.7 ms, System: 310.2 ms] Range (min … max): 1.092 s … 1.113 s 10 runs Alleviates #59884. For the most part each commit stands on its own. Timings are on warm filesystem cache. r? @eddyb

bors · 2019-06-23T22:50:52Z

☀️ Test successful - checks-travis, status-appveyor
Approved by: eddyb
Pushing de7c4e4 to master...

rust-highfive assigned eddyb Jun 21, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 21, 2019

Mark-Simulacrum force-pushed the tidy-fast branch from a0e1e9e to 2a0d1cc Compare June 22, 2019 00:31

eddyb approved these changes Jun 23, 2019

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 23, 2019

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jun 23, 2019

Mark-Simulacrum added 11 commits June 23, 2019 09:09

Make tidy quieter by default

2b6371d

Cache Regex's

5f1da8e

Use walkdir crate

e17b02d

It's more efficient than the fs::read_dir API

Stop calling format! to check feature gate

5c33c3e

Pass contents and DirEntry to walkers

c113a37

Move file-reading into walker loop

38f0b90

Utilize entry.metadata over fs::symlink_metadata

d619e44

Skip querying each ignore directive if none in file

7dd7c0f

Use Path/PathBuf directly instead of through path::

ebbc662

Collect features only once

6c5c78d

Exit early from feature search if no features in file

777951c

Mark-Simulacrum force-pushed the tidy-fast branch from 2a0d1cc to 777951c Compare June 23, 2019 13:10

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 23, 2019

Centril mentioned this pull request Jun 23, 2019

Rollup of 5 pull requests #62080

Closed

bors added the merged-by-bors This PR was explicitly merged by bors. label Jun 23, 2019

bors merged commit 777951c into rust-lang:master Jun 23, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Speed up tidy #62037

Speed up tidy #62037

Mark-Simulacrum commented Jun 21, 2019

rust-highfive commented Jun 21, 2019

eddyb commented Jun 23, 2019

bors commented Jun 23, 2019

bors commented Jun 23, 2019

Mark-Simulacrum commented Jun 23, 2019

bors commented Jun 23, 2019

bors commented Jun 23, 2019

bors commented Jun 23, 2019

Speed up tidy #62037

Speed up tidy #62037

Conversation

Mark-Simulacrum commented Jun 21, 2019

rust-highfive commented Jun 21, 2019

eddyb commented Jun 23, 2019

bors commented Jun 23, 2019

bors commented Jun 23, 2019

Mark-Simulacrum commented Jun 23, 2019

bors commented Jun 23, 2019

bors commented Jun 23, 2019

bors commented Jun 23, 2019